An improved framework of GPU computing for CFD applications on structured grids using OpenACC

This paper is focused on improving multi-GPU performance of a research CFD code structured grids. MPI and OpenACC directives are used to scale the up 16 GPUs. shows that using P100 GPUs V100 can be 30$\times$ 70$\times$ faster than Xeon CPU E5-2680v4 cores for three different test cases, respectively. A series issues related scaling multi-block addressed by applying various optimizations. Performance optimizations such as pack/unpack message method, removing temporary arrays arguments procedure calls, allocating global memory limiters connected boundary data, reordering non-blocking I\_send/I\_recv Wait reducing unnecessary implicit derived type member data movement between host device use GPUDirect improve compute utilization, throughput, asynchronous progression in modern programming features.